Sorry, can't make it.
Decline the offer. To have both of them visiting your congregation means that you guys are probably the host for an upcoming Circuit Assembly. If you were once an elder, then I would expect that they will guilt bomb you into getting your act together. If you were never serving, then they probably asked the BOA for a list of those would might "benefit" from a shepherding call. If your wife and kid have no issue with your current situation, then I would suggest you tell them you can't make it. You can probably defray any backlash by approaching both of them at the meeting and thanking them personally.
As Doc has mentioned above, if you have issues with the ORG, these type of visits can go haywire real quick.
SOP